Convexity, Detection, and, Generalized f-divergences

The goal of multi-class classification problem is to find a discriminant function that minimizes the expectation of 0-1 loss function. However, minimizing 0-1 loss directly is often computationally intractable and practical algorithms are usually based on convex relaxations of 0-1 loss, say Φ, which is called the surrogate loss. In many applications, the covariates are either not available directly but are received after passing through a ‘dimension reducing’ quantizer or are deliberately transformed using a ‘feature selection’ stage to achieve a better interpretation. In experimental design, one fundamental question is that how to select the a quantization procedure that decreases our optimal Φ risk, RΦ(Q) the most. Another fundamental question here is to find all the surrogate losses Φ that are universally equivalent to the 0-1 loss functions, in the sense that, Φ and 0-1 loss induce the same ordering on the optimal risk of quantizers.